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Abstract 

As  our  Armed  Forces  transform,  assisted  by  the  Brigade 
Combat  Team  (BCT)  Modernization  effort,  the  use  of 
Models  and  Simulations  (M&S)  becomes  more  crucial  in 
supporting  major  Department  of  Defense  and 
Congressional  decisions,  given  limited  resources  and 
strategic  constraints.  The  Army’s  program  leading  the 
transformation  from  2003  to  2009  was  Future  Combat 
Systems  (FCS)  with  14  systems  +  The  Soldier  +  their 
network.  For  Phase  1  (IP1)  Spinout  1  (SOI)  Technical 
Field  Test  (TFT),  three  live  systems  (i.e.,  Non-Fine  of 
Sight-Faunch  System  (NFOS-FS),  Unattended  Ground 
Sensor  (UGS),  and  B-Kits)  participated  and  were  tested  in 
2008  via  a  slice  of  the  Current  Force  (CF)  BCT  structure. 
To  ensure  realistic  operational  context,  a  M&S  System-of- 
Systems  (SoS)  level  federation  was  developed  providing 
virtual  and  constructive  simulation  capabilities  that 
enabled  holistic  testing  with  complex  integration  among 
all  entities  in  a  distributed  Five- Virtual-Constructive 
(FVC)  environment.  This  included  interfaces  to  live 
entities  and  instrumentation  via  tactical  messages,  and 
constructive  representation  of  platforms,  vehicles,  and 
terrain.  The  M&S  federation  also  provided  test  control, 
data  collection,  and  live  range  interactions. 

The  foundation  for  the  Accreditation  process  of  the  SOI 
TFT  federation  was  the  Verification,  Validation  and 
Accreditation  (VV&A)  Overlay  to  the  DoD  High  Fevel 
Architecture  (HFA)  Federation  Development  and 
Execution  Process  (FEDEP).  A  three-phased  V&V 
process  was  used  that  provided  component  level  V&V, 
initial  Federation  V&V  via  multiple  M&S  integration 
events,  with  the  final  events  performed  in  the  FCS  Mobile 
Node.  The  SOI  TFT  M&S  Federation  developed  by  the 
Cross  Command  Collaboration  Effort  (3CE)  was  part  of 
the  “common  SOI  M&S/tools  federation  solution”  for  all 
SOI  test  events. 


The  Acceptability  Criteria  were  developed  via  an  iterative 
process,  involving  the  Test  Manager  and  all  VV&A  Teams, 
that  began  with  identifying  Assessment  Objectives  associated 
with  M&S  for  IP1  SOI  TFT  that  can  be  traced  back  to  the 
FCS  requirements/capabilities  and  TRADOC  Operational 
and  Organizational  documentation.  Once  the  Acceptability 
Criteria  and  Metrics  (ACM),  9  and  25  respectively,  were 
approved  several  traceability  analyses  were  conducted  with 
ACMs  thoroughly  examining  the  2 1 3  M&S  requirements  and 
M&S  intended  uses  developed  by  the  Test  Manager.  The 
results  helped  to  form  a  solid  foundation  for  accreditation 
assessment  providing  focus  in  building  the  body  of  V&V 
evidence  and  the  accreditation  methodology. 

To  verify  M&S  requirements  and  objectives  the 
Accreditation  Team  worked  closely  with  V&V  Teams,  Test 
Manager,  3CE  and  all  M&S  component  developers  during 
all  Integration  Events  to  understand  functions  and 
capabilities  of  TFT  M&S  Federation  and  each  individual 
component,  data,  and  support  tools.  This  invaluable 
experience  provided  insight  on  data  collection  and  format, 
terminologies  used,  and  expectations. 

Accreditation  assessments  were  conducted  to  support 
preliminary  and  final  Test  Readiness  Reviews  (TRR).  The 
Accreditation  Assessment  rating  used  was  a  5-point 
satisfaction-risk  table  developed  by  Accreditation  Team. 
By  the  final  TRR  all  requirements  were  verified,  all  results 
validated,  and  all  accreditation  metrics  and  criteria  were  met 
successfully  demonstrating  an  Unconditional  Accreditation 
of  the  SO-1  TFT  M&S  Federation  in  support  of  the  SOI 
TFT  Runs  for  Record. 
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Validation,  Accreditation,  VV&A,  Federation,  System  of 
Systems,  FVC,  Five  Virtual,  Constructive 
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1.  Introduction  and  Background 

As  our  Armed  Forces  transform,  assisted  by  the  Brigade 
Combat  Team  (BCT)  Modernization  effort,  the  use  of 
Models  and  Simulations  (M&S)  becomes  more  crucial  in 
supporting  major  Department  of  Defense  and 
Congressional  decisions,  given  limited  resources  and 
strategic  constraints. 

The  U.S.  Army’s  program  leading  the  transformation  from 
2003  to  2009  was  Future  Combat  Systems  (FCS)  with  14 
systems  +  The  Soldier  +  the  supporting  network.  For 
Phase  1  (IP1)  Spinout  1  (SOI)  Technical  Field  Test 
(TFT),  three  live  systems  (i.e.,  Non-Line  of  Sight-Launch 
System  (NLOS-LS),  Unattended  Ground  Sensor  (UGS), 
and  B-Kits)  participated  and  were  tested  in  2008  via  a 
slice  of  the  Current  Force  (CF)  BCT  structure.  To  ensure 
realistic  operational  context,  an  M&S  System-of- Systems 
(SoS)  level  federation  was  developed  to  provide  virtual 
and  constructive  simulation  capabilities  that  enabled 
holistic  testing  with  complex  integration  among  all  entities 
in  a  distributed  Live- Virtual-Constructive  (LVC) 
environment.  The  federation  included:  interfaces  to  live 
entities  and  instrumentation  via  tactical  messages; 
constructive  representations  of  platforms,  vehicles,  and 
terrain;  test  control,  data  collection,  and  live  range 
interactions. 

2.  Constraints 

When  Army  Leadership  decided  that  the  M&S  Federation 
supporting  the  SOI  TFT  required  Accreditation,  there 
were  many  constraints  regarding  the  SOI  test  events 
already  in  place.  Although  the  Accreditation  Team  was 
not  involved  from  the  very  beginning,  the  team  was  able 
to  work  within  the  “constraints”  framework  and  make  a 
positive  impact.  Some  of  the  “constraints”  in  place  were: 

1.  VV&A  roles  and  responsibilities 

A  different  organization  was  responsible  for  each  test 
event;  therefore,  a  different  organization  was  responsible 
for  accrediting  the  M&S  that  supported  a  given  test  event. 

2.  Essential  FCS  documents  (e.g.,  plans  and 
requirements  documents) 

The  initial  draft  of  V&V  Plan  was  started  before  the 
Accreditation  Team  came  on  board. 

3.  Planning  and  Execution  Test  Schedule 
Less  than  one  year  was  scheduled  to  conduct  the  SOI 
TFT  Accreditation  Assessment.  The  Accreditation 
Team’s  involvement  began  after  everyone  else. 


4.  There  were  307  Test  Requirements  generated  by 
Test  Manager,  the  Lead  System  Integrator  (LSI), 
the  contractor  that  executed  the  SOI  TFT  event 

Once  the  Acceptability  Criteria  (AC)  and  Metrics  (ACMs), 
were  finalized  and  approved,  they  were  analyzed  against  the 
307  Test  Requirements2  .  The  Accreditation  Team  identified 
94  as  being  necessary  for  the  test  event  but  outside  the 
VV&A  scope  leaving  213  identified  as  M&S  Requirements. 

5.  M&S  Federation,  M&S  support  tools,  and  data 
integrated  by  the  Cross  Command  Collaborative 
Effort  (3CE) 

The  3CE  Group  was  responsible  for  defining  “the  M&S 
Common  Solution”  for  all  the  SOI  Events  (i.e.,  TFT, 
FDT&E,  LUT  and  Train-up).  The  3CE  addressed  M&S 
issues  and  requirements,  defined  the  M&S  architecture,  and 
performed  M&S  integration  and  testing. 

3.  Accreditation’s  Team  Approach  Used  to 
Accredit  the  M&S  Federation 

The  approach  followed  was  the  Federation  Development  and 
Execution  Process  (FEDEP),  but  was  tailored  to  fit  the  needs 
and  requirements  of  the  BCT-M  Program.  As  stated  earlier, 
3CE  developed  the  M&S  Federation  and  also  led  the 
integration  events. 

The  Acceptability  Criteria  were  the  “heart  and  soul”  of  the 
M&S  Federation  Accreditation  because  they  address  the 
suitability  of  the  M&S  Federation  for  the  current  intended 
specific  use  and  guided  the  collection  of  V&V  artifacts. 
Before  developing  the  Acceptability  Criteria,  a  solid 
understanding  of  the  supported  test  event  was  acquired.  The 
Acceptability  Criteria  were  developed  based  on  three 
converging  elements: 

1 .  Program  and  Technical  Path  -  The  Program  factor 
was  based  on  the  test  Assessment  Objectives  (AO)  that  were 
traced  through  System  of  Systems  (SoS)  Engineering  and 
Integration  (SSEI)  Integrated  Product  Team  (IPT) 
documentation  back  to  FCS  Program  requirement 
documents.  Specifically,  the  31  AOs  for  the  SOI  TFT  were 
reviewed  and  analyzed.  Only  10  system-oriented  AOs  were 
identified  as  being  associated  with  M&S. 


2  The  SOI  TFT  Test  Requirements  were  identified  in  the 
Technical  Requirements  Alignment  Matrix  (T-RAM),  which 
showed  the  link  to  the  Objective  System  Test  Requirement 
Document  (OSTRD)  via  a  Requirements  ID  (RID)  number. 
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Accreditation  Assessment  summary  format; 


The  Technical  factor  was  based  on  the  307  Test 
Requirements,  developed  by  the  Test  Manager,  to  support 
the  test  event.  The  list  of  307  Test  Requirements  were 
analyzed  against  the  initial  set  of  Acceptability  Criteria 
Metrics,  and  many  were  identified  as  not  applicable  (NA) 
or  outside  the  VV&A  scope  though  necessary  for  the  test 
event.  Therefore,  out  of  the  307  Test  Requirements,  213 
were  determined  to  be  “true”  M&S  Requirements.  The 
analyses  conducted  on  the  AOs  and  Test  Requirements 
helped  to  refine  and  shape  the  Acceptability  Criteria  and 
Metrics. 

2.  Policy  Path  -  The  Policy  Path  was  based  on 
DoD,  the  Army,  and  Command-oriented  policies.  The 
specific  policy  guides  supporting  this  test  event  were: 
DoD  5000.61  (DoD);  DA  PAM  5-11  (Department  of  the 
Army);  and  ATEC  Reg  73-21  (HQ  Army  Test  and 
Evaluation  Command).  From  these  policies,  three 
additional  and  essential  criteria  were  identified: 
configuration  management,  data  pedigree  (i.e.  obtained 
from  an  authoritative  source);  and  federate  components 
communication  amongst  themselves. 

3.  Intended  Use  Cases  Path  -  The  Intended  Use 
Cases  Path  was  based  on  the  Detailed  Test  Plan  that 
identified  the  test  cases  to  be  executed  during  the  test. 
Before  finalizing  the  Acceptability  Criteria  and  Metrics, 
the  test  cases  were  reviewed  and  analyzed  against  the 
Intended  Uses  looking  in-depth  at  the  specific  Use  Cases 
or  test  scenarios  and  how  the  M&S  was  applied. 

From  these  three  converging  paths,  three  Traceability 
Analyses  Matrices  were  developed: 

1.  Acceptability  Criteria  Metrics  (ACMs)  to  the 
Assessment  Objectives  (AOs); 

2.  ACMs  to  the  307  M&S  Test  Requirements, 
defined  by  the  Test  Manager;  and 

3.  ACMs  to  the  Intended  Uses  (IUs). 

From  these  traceability  analyses  conducted  six  benefits 
were  derived  as  follows: 

1 .  Helped  to  refine  and  shape  the  Acceptability 
Criteria  (AC)  and  the  Metrics  (ACMs); 

2.  Resulted  in  identifying  213  requirements  as 
being  “true”  M&S  requirements; 

3.  Provided  the  essential  underpinning  for  the 
Accreditation  Assessment  methodology, 
including  how  best  to  present  the  traceability  and 
support  of  the  V&V  evidence  in  a  top-level 


4.  Served  as  useful  template  to  present  top-level 
Accreditation  Assessment  summary  in  the  most  test 
related  context,  from  the  customer’s  and  user’s 
perspective,  supported  by  the  M&S  requirements 
traceability  matrix; 

5.  The  ACMs  and  the  Test  Requirements  Traceability 
Analysis  were  deemed  “priceless”  by  both  the 
V&V  and  Accreditation  Teams.  The  Traceability 
Matrices  were  the  “backbone”  of  the  Accreditation 
Assessment  methodology  providing  a  solid 
foundation  to  explore  and  present  the  data  from 
multiple  perspectives:  Acquisition  Decision  maker, 
Test  Manager,  Test  evaluator  and  M&S  SME. 

6.  These  rigorous  Accreditation  activities  allowed  9 
Acceptability  Criteria  and  25  Acceptability  Criteria 
Metrics  to  be  approved  for  the  SOI  TFT  M&S 
Federation. 

Some  of  the  V&V  activities  proved  to  be  of  great  help  to  the 
Accreditation  Team  including  the  development  of  two 
standardized  forms: 

•  V&V  M&S  Federation,  federates,  components, 
and  support  tools  Description  Form  - 
standardized  format  of  V&V  documentation 
that  described  in  detail  what  the  M&S 
component  brought  to  table  and  how  it  would 
be  used.  These  forms  were  completed  by  the 
component  developers. 

•  V&V  Artifact  Fog  Form  -  described  the  M&S 
requirement  in  detail,  data  collection  events, 
verification  method  &  status,  ACMs  supported, 
procedure  and  results.  The  log  forms  were 
completed  by  the  V&V  Team.  The  form 
provided  standardized  format  as  part  of  V&V 
process  and  documentation  supporting  the 
M&S  Accreditation. 

4.  Results 

The  SOI  TFT  M&S  Federation  was  part  of  the  “common 
SOI  M&S/tools  federation  solution”  for  all  SOI  test  events. 
However,  each  test  event  had  different  responsible 
organizations,  test  objectives,  applications,  and  M&S 
requirements.  Based  on  these  differences  and  required 
updates  to  the  SOI  M&S  Federation  to  support  subsequent 
tests,  per  AR  5-11,  a  separate  accreditation  of  the  federation 
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was  required  for  each  intended  use,  in  this  case,  for  each 
test  event. 

The  Accreditation,  V&V  and  Independent  V&V  (IV&V) 
Teams  participated  in  the  SOI  TFT  Federation  Integration 
Events  (IE)  in  order  to  understand  the  federation  and  its 
components  and  to  verify  the  M&S  requirements  and 
objectives.  The  IEs  were  also  used  to  validate  the  M&S 
requirements  and  Assessment  Objectives  of  the  SOI  TFT 
M&S  Federation.  The  IEs,  which  took  place  from  October 
2007  to  January  2008,  served  as  the  final  V&V  events  to 
ensure  that  all  problems  identified  during  earlier  IEs  were 
fixed  and  to  demonstrate  that  the  SOI  TFT  Federation 
functioned  as  expected  in  support  of  the  SOI  TFT.  Due 
to  software  delays,  the  final  acceptance  event  was  the  TFT 
Dry  Run  which  took  place  in  February  2008,  just  days 
before  SOI  TFT  execution. 

The  Preliminary  Test  Readiness  Review  (PTRR)  was  held 
on  25  January  2008  and  the  overall  recommendation 
presented  by  the  Accreditation  Team  was  to  continue  with 
TFT  as  scheduled,  because  MET  was  anticipated  by  the 
Test  Readiness  Review  (TRR). 

Additional  testing  was  completed,  and  data  were  collected 
during  the  TFT  Dry  Runs,  13-19  February  2008. 

The  TRR  was  held  on  22  February  2008.  The  overall 
accreditation  status  at  the  time  of  the  TRR  was  “Met”  for 
the  following  reasons: 

•  All  requirements  were  verified; 

•  All  results  were  validated;  and 

•  All  Acceptability  Criteria  were  Met. 

There  were  no  outstanding  Accreditation  issues.  All 
M&S  capabilities  needed  for  SOI  TFT  were  successfully 
demonstrated  prior  to  the  TRR.  Acceptability  criteria 
metrics  were  sufficiently  met  to  support  a  favorable 
accreditation  recommendation  prior  to  SOI  TFT  TRR. 

The  overall  recommendation  presented  by  the 
Accreditation  Team  was  an  Unconditional  Accreditation 
of  the  SO-1  TFT  M&S  Federation  in  support  of  the  SO-1 
TFT  Runs  for  Record.  The  “Accreditation  Decision 
Memorandum  for  Record  (MFR)”  dated  22  Feb  2008 
officially  authorized  the  use  of  the  SOI  TFT  Federation  in 
the  SOI  TFT  Test  for  Record. 


5.  Lessons  Learned 

The  lessons  learned  fall  into  three  categories:  operational  / 
implementation;  process;  and  the  combination  of  the  two 
categories. 

Operational  /  Implementation 

■  Participate  in  Integration  Events  and  Dry  Runs, 

■  Develop  Traceability  Matrices, 

■  Develop  relevant  Acceptability  Criteria  and  Metrics, 

■  Expand  Intended  Uses  to  include  Metrics,  and 

■  Leverage  resources  and  VV&A  documentation  where 
possible. 

Process 

■  Start  early  -  planning  &  working  with  complete  VV&A 
Team, 

■  Engage  all  stakeholders  early, 

■  Coordinate  the  staffing  and  approval  requirements, 

■  Develop  Accreditation  standardized  formats, 

■  Refine  V&V  standard  forms  to  better  support 
Accreditation,  and 

■  Establish  an  Accreditation  process  that  can  used  in 
subsequent  tests,  if  possible. 

Both  Categories:  Operation  /  Implementation  and  Process 

■  Encourage  Team  work,  and 

■  Use  a  collaborative  environment  -  to  make  information 
sharing  and  internal  reviews  easier. 


6.  Sharing  VV&A  Forms/Formats  Developed 

The  SOI  TFT  VV&A  Team  developed  5  forms  and  /or 
formats  that  were  proven  useful  regarding  V&V  and 
Accreditation  activities.  Further  the  usefulness  of  these 
forms  and  formats,  listed  below,  have  been  substantiated  via 
support  to  other  test  and  demonstration  events: 

1.  M&S  Federation,  federates,  components,  and 
support  tools  form 


V&V  Form 

Description  for  M&S  f 


2.  V&V  Artifact  Log  form 


V&V  Activity  Log  - 
Blank  Form,  doc 
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3.  Acceptability  Criteria  Traceability  to  M&S 
Requirements  Matrix 


Traceability  Analysis 
Matrix  formxls 


4.  Accreditation  Assessment  (AA)  Summary 

l!?  ^ 

M&S  Accreditation 
Assessment  Summary 


5.  AA  M&S  Intended  Use  Summary 


Accreditation 
Assessment  Intendec 

7.  Conclusion 

This  paper  presents  the  Accreditation  activities  conducted 
on  the  M&S  Federation  supporting  the  SOI  TFT  to  ensure 
realistic  operational  context.  An  M&S  System-of- Systems 
(SoS)  level  federation  was  developed  to  provide  virtual 
and  constructive  simulation  capabilities  that  enabled 
holistic  testing  with  complex  integration  among  all  entities 
in  a  distributed  Live- Virtual-Constructive  (LVC) 
environment. 

Specifically,  this  paper  discussed  how  the  Accreditation 
Team: 

1 .  Dealt  with  constraints ; 

2.  Developed  the  Accreditation  Criteria  and  Metrics 
based  on  the  convergence  of  three  paths:  Program 
and  Technical  Path;  Policy  Path;  and  Intended  Use 
Cases  Path; 

3.  The  three  Traceability  Analyses  conducted  and  their 
benefits; 

4.  Identified  two  standardized  forms  developed  by  the 
V&V  Team  that  were  found  to  be  most  helpful; 

5.  Summarized  the  results  of  the  Accreditation 
Assessment  to  support  readiness  reviews; 

6.  Provided  the  summarized  Accreditation  results 
presented  at  readiness  reviews; 

7.  Identified  lessons  learned;  and 


8.  Provided  blank  forms  and  formats  that  would  be  useful 
for  the  conduct  of  V&V  and  Accreditation  activities 
supporting  any  test  or  demonstration  events. 
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Background:  FCS  Program  Overview 


(Program  transitioning  to  Brigade  Combat  Team  -  Modernization  (BCT-M)) 


The  FCS  program,  considered  the  core  building  block  of  the  Army's  future  force,  consists  of  the  following  elements: 

■  The  network  (information  and  communications)  + 

■  14  individual  combat  systems  including  manned  and  unmanned  systems  + 

■  The  soldier . 

Because  all  of  the  constituent  parts  of  the  FCS  program  are  viewed  as  systems  in  themselves  --  including  the  14  sub-systems,  the 
network  itself,  and  even  the  individual  soldier  --  it  is  commonly  referred  to  as  the  "14+1+1"  system  or  a  "system  of  systems.” 


Constraints 


■  VV&A  roles  and  responsibilities 

■  Essential  FCS  documents  (e.g.,  DTP,  ORD/CDD,  etc.) 

■  Planning  and  Execution  Test  Schedule 

■  307  Test  Requirements  generated  by  Test  Manager,  the 
Lead  System  Integrator  (LSI),  which  is  the  contractor  that 
executed  the  SOI  TFT  event 

•  213  M&S  Requirements 

■  M&S  Components  -  federation,  support  tools  and  data 
integrated  by  Cross  Command  Collaborative  Effort  (3CE) 

■  Accreditation  Team  involvement  began  after  everyone  else 


VV&A  Roles  and  Responsibilities 


V&V  Agent 
(Lead) 

IV&V  Agent 
(Lead) 

Accred.  Auth. 
[Designee] 

Accred.  Agent** 

(Lead) 

[Designee] 

SO-l 

TFT 

LSI  V&V  Team 
(LSI) 

MSO  IV&V 
(LSI) 

PM  FCS 
[GO] 

PM  FCS 

MSO 

(MSO) 

[Debra  Ridgeway] 

SO-l 

FDT&E 

TRADOC 

LSI  V&V  Team* 

MSO  IV&V* 
(LSI) 

CG  TRADOC 
[GO] 

TRADOC 

FFID  M&S 

SO-l 

Pre-LUT 

ATEC 

(OTC) 

LSI  V&V  Team* 

MSO  IV&V* 
(LSI) 

CG  ATEC  *** 

[GO] 

ATEC 

AST 

(AEC) 

[OTC] 

IMT-1 

LSI  V&V  Team 

MSO  IV&V 
(LSI) 

PM  FCS 

GO 

[MSO] 

N/A 

Exp.  2.1 

LSI  V&V  Team 

MSO  IV&V 
(LSI) 

PM  FCS 
[GO] 

N/A 

*  Support  Agent’s  process  as  requested  by  MSO 

**  Accreditation  Authority  designates  accreditation  agent  (organization)  and  lead  staff 
***  ATEC  will  certify  M&S  for  LUT 


Note:  In  cases  where  ATEC  plans  to  use  data  from 
other  than  the  LUT  event,  CG  ATEC  will  be  afforde<fcthe 
opportunity  to  accredit  the  M&S  used  to  support  sifth 
events. 


W&A  Planned  for  Execution 
Schedule  for  SOI  TFT 


Revision  16  17  Sep  07 


3QTR  FY  07 

4QTR  FY07 

1QTR  FY08 

2QTR  FY8 

March 

April 

May 

June 

July 

August 

September 

October 

November 

December 

January 

February 

r  Start 


£ 


I 


Acer  Criteria 


6/1/07 


I 


in/i«;/n7 


v&v 


IV&V 


I 


AcA  Plan 


Accreditation 


* 


8/28 

7! 


Accreditation  Preparation  /  Monitor  V&V  Activities 


i 


£ 


V&V  Plan  D786-123£3-1 

T 


r  Certifications 
Threat 


I 


4E-#3- 


IE  #4 


Terrain  DB 
IE  #5  lnAa  IE  #6 

ioS 


± 


V&V  Component 


•  WS  Perf  Data 

•  TSP 


_ 


jAd  SoS  AssessrAj 


nt/E^cution 


Component  Assessment  #1 


E 


#2 


j/k  i 

£  i 

k  A 

£  A 

£  J 

£  A 

7/20  B 

£i 

IV&V  Plan 


w////y////y///7mL 


1/25/08  1/28/08 


2/22/08 


Holiday  Periods 


I 


Concitional  Un-Conditional 


I 


1/p  8  2/M 

"V/////A 


y  IE  #10? 

Terrain  DB 
Final 


Acer  Support 


Wf/Z/s. 


IV&V  Component  and  SoS  Assessment/Execution 


I 


£ 


IV&V  Re^rt 


F 


£ 


Acer  Support 
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LEGEND 


▲  Program  AY7  item  Start/Finish  Aw  Item  Start/Finish  aY7  Draft  (Unapproved)  avt  M&S  Delivery  A  SOI  M&S  ^  n  .  A  No  Impact 
w  Inchstone  AV  (Planned)  AT  (Actual)  AV  Planned/Actual  AV  Planned/Actual  A  Integration  Event  V  uata  A  jQ  soi  TFT 


SOI  TFT  M&S  Federation,  Data  and  Support  Tools* 


M&S 

Category 

M&S  Tool  Name 

Oommon  Name  / 
Acronym 

Provider 

Intended  Use 

M&S 

Federation 

Objective  Force-One  Semi-Automated 
Force  Objective  System 

O  n  e  S  AF 

PM 

O  n  e  S  AF 

CGF:  .2D  Display  of  Battlefield 
(Simulated  and  live);  Scenario 
generation. 

Command  and  Control  Adapter 
(part  of  Objective  Force-One  Semi- 
Automated  Force  Objective  System  ) 

C2  Adapter 
■(part  of  GneSAF) 

PM 

O  n  e  S  AF 

Converts  GneSAF  internal  format 
messages  to  tactical  J\/MF;  Converts 
tactical  J VrM F  messages  to  OneSAF 
internal  format 

Extensible  (C-41)  Instrumentation  Suite 

ExC  1 S 

OTC 

Interface  to  live  AFATDS  required  for 
NLOS-LS  IFire  Control  Solution: 

Simulated  NLOS-LS  PAM  fly  out 

M&S  Data 

WS M R  Te rra in  Data ba s e 

WS  M  R  TDB 

TEC 

Digital  representation  ofthe  event 
execution  area.  Terrain  of  WS  MR  for 
common  use  by  the  federation. 

Federation 

Support 

Tools 

Digital  Collection,  Analysis,  and 

Review  System  II  / 

Reconfigurable  Intelligent 
Instrumentation  to  Collect,  Simulate 
and  Stimulate 

D  CARS  / 

RICS 

EPG 

Data  collection,  monitoring,  near 
real-time  review,  storage,  and  AAR 
capability 

STARSHIP  II 

(Stars  hip  /  StarGen) 

Starship 

StarGen 

PM  ITTS 

Monitor  and  control  health  and  status  of 
(RICS >2  and  H LA  federates 

Modeling  Architecture  for  Technology, 
Research  and  EXperimenation 
-  Federation  Object  Model 

Run  Time  Infrastructure 

MATREX 
-  FOM 
-  RTI 

RDECOM  / 
MATREX 

Controlling  mechanism  for  H LA.  federation 

3  Dimension  Visualization 

3  DViz 

(Part  of  MATREX) 

RDECOM  / 
C  E  RD  E  C  / 
NVESD 

Provides  ground  truth  with  near  real-time 

3D  visualisation  of  battlefield  for  Live  and 
Constructive  entities. 

Acquisition  Reporting  and  Display 

Syst  e  m 

ARDS 

WS  M  R 

Ground  truth  for  live  assets  instrumented 
with  ARDS  pods. 

Fort  Bliss  Homestation 

Instrumentation  Test  System 

Fort  Bliss  HITS 
DCNC 

EXC  ON 

VR  Exchange 

EPG 

Provide  ground  truth  of  live  vehicles  to 

H LA  Federation  via  instrumentation. 

Also  provide  AAR  capability. 

Test  Conduct  Reporting  System 

TCRS 

EPG 

Test  management  and  data 
col  1  ect  i  o  n/re  po  rt  i  ng 

Orin  Test  Talk 

T  estTalk 

EPG 

Test  situational  awareness 

C4ISR  Test  Took  Kit  -  Data  Reduction 
and  Analysis 

CTTK-D  RA 

EPG 

Near  real-time  and  post-test  analysis 
and  reduction  of  data  collected  from 

FCS  platforms  instrumented  with  the 

EPG  Data  Collection  Suite. 

Federation  integrated  by  Cross  Command  Collaborative  Effort  (3CE)  and  was  part  of  the  “common  SOI  M&S  federation  /  tools  solution 


Notional  SO-1  LVC  Operational  View 

Live  -Virtual-  3onstructive  Integration 


C3 


1/1  AD 


MCS 


Joint  /  ISR 
Feeds 


Mobile  Node 


■  ■ 


2E*  Live 


UGS 


I  MCS*  ; 
<  ASAS  L  \  .. 

IbCS£.***\ 

^f^TDS 


r 


FBCB2 


B  Kit 


MS 


RPWS  + 


M  LES 


FFID  Simr*Rac 


Constructive  Wrap-ar 


AAR 

DCARS 


Acronyms  list  in  Back-ups 


Accreditation  Team’s  Approach 

Used  to  Accredit 
SOI  M&S  Federation 


Practices,  Methodologies  and  Procedures 


Determining  Acceptability  Criteria 


gSfSiJechnical 


Documented 

Program 

Requirements 

TRADOC  Opn’l  &  Org 


AOs  Associated  w /  M&S 

T-UGS  -  3  AOs 

SO1-1100 

Path  S01'1200 

-  1,1  SO1-1300 


SSE&I  Docs 

Requirements 
Specifications 
&  Verification 
Plans 


Assessment 

Objectives 

(AOs) 


U-UGS  --  4  AOs 

SO1-2100 
SOI -2200 
SOI -2300 
SOI -2400 

NLOS-LS  -  2  AOs 

SO1-3100 
SOI -3200 

Level  1  Fusion  -  1  AO 

SOI -71 00 


Acceptability 

Criteria 


(0 

a. 

> 

o 

o 

CL 


Source:  Doc#  D786-1 1 925-1  Draft  -  dated  5Feb07 

System  of  Systems  (SoS)  Engineering  and  Integration  (SSEI)  Integrated  Product  Team  (IPT) 
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Approved  FCS  SOI  TFT  Acceptability  Criteria  and  Metrics 


AC# 

Acceptability  Criteria  Title 

SOI  TFT  Acceptability  Criteria  (AC)  Descriptions 

ACM  # 

SOI  TFT  AC  Metrics 

1 

Terrain  representations  are  at 
required  resolution. 

This  federation  must  provide  consistent  realistically  and  doctrinally  correct  simulated 
terrain  and  environment  representations  of  the  test  site  terrain  as  defined  in  the 
Integrated  Phase  1  (1P1)  Detailed  Test  Plan  for  Spin  Out  1  (SOI  DTP)  TFT  dated 

6  March  2007;  System  Evaluation  Plan  (SEP)  for  FCS  SOI  dated  25  January  2007: 
and  FCS  Test  and  Evaluation  Master  Plan  (TEMP)  dated  7  April  2007  to  support  the 
integration  of  the  Live  and  Constructive  (LG)  environment  across  all  involved  SOI  TFT 

or-.+  i+io^^^ 

ACM1-1 

Terrain  scalability 

ACM1-2 

Terrain  3-4D  display 

ACM1-3 

Terrain  representation 

2 

Configuration  Managed  Federation. 

This  federation  for  SOI  TFT  must  ensure  operational  consistency  regarding  its 
functional  use  and  capability,  and  physical  characteristics,  as  described  in  AR  5-11. 
Chapter  6.  Configuration  Management,  which  includes,  but  not  limited  to.  purpose, 
assumptions,  limitations  etc.  that  must  be  documented  and  properly  maintained  after 
3CE  hand-off  to  test  event  lead. 

ACM2-1 

Configuration  identification 

A  C  M  2  -2 

Configuration  control 

ACM2-3 

Configuration  status  accounting 

AC  M2  4 

Configuration  history 

3 

The  simulated  entities  must  perform 
and  interact  as  required  to  support 

the  SOI  TFT. 

me  simulated  entities  l,e.g..  vemcies.  soldiers,  etc.}  tor  the  u-ntTusOe 

realistic  representations,  correctly  identified  and  defined  in  the  DTP  and  TEMP  to 
include  proper  interactions  between  simulated  and  live  entities,  correlated  terrain  and 
must  meet  the  standards  as  defined  in  the  Technical  Requirements  Alignment 

Matrix  I'^^AM^^ 

ACM3-1 

ID  of  simulated  entities  -3.  objects 

AC  M3 -2 

Representation  of  OF,  threat  entities 
a  SOI  systems 

4 

All  federation  input  data  must  be 
identified  and  provided  as  required 
for  SOI  TFT  and  certified  for  the 
intended  use  by  authoritative 
source(s]. 

Data  used  in  models  and  simulations,  as  part  of  the  federation  supporting  the  SOI 
TFT.  must  be  verified,  validated,  and  certified  (W&C)  for  intended  use.  as  described 
in  AR  5-11.  ATEC  PAM  73-21  and  SEP.  Data  must  be  obtained  from  Government 
Furnished  Equipment  (GFX)  or  from  the  FCS  One  Team  Partners  (OTPs)  responsible 
for  the  provision  of  the  articles  under  test  in  the  TFT. 

ACM4-1 

V-&V  of  input  data 

ACM4-2 

Data  certification  letter 

5 

SOI  TFT  Federation  must  run  in  real¬ 
time  to  support  live  test. 

Simulated  entities  must  operate  in  real-time,  appear  correctly  on  the  live-systems' 
and  constructive-systems'  displays  from  the  entities  generated  by  individual 
federates  and  the  federation  as  a  whole  in  order  to  produce  and  update  entities  at  the 
same  rate  as  the  systems  under  test. 

ACM  5-1 

No  message  delay 

ACM  5 -2 

No  ground  truth  delay 

6 

The  SOI  TFT  Federation  must 
su  p  p  o  rtiri  nte  rope  rate  with  current 
force  battle  command  networks. 

The  TFT  Federation  must  be  able  to  communicate/  interoperate  with  existing  live  and 
simulated  Army  battle  command  /  systems  networks.  Federation  entities  must 
accurately  send,  receive,  acknowledge,  and  display  required  tactical  messages.  In 
addition,  the  Federation  must  be  able  to  simulate  the  Non-Line-of-Sight  -  Launch 
System  (NLOS-LS)  as  required  by  the  TFT.  The  Federation  must  be  able  to  execute 
timely  NLOS-LS  fire  missions  based  on  receipt  of  observed  targetable  data. 

ACM6-1 

Co  mono  &  interoperate  w/  existing 
Army  Fire  controls 

ACM  6 -2 

Fire  mission  -  PAM  flyout 

ACM6-3 

Tactical  message  -  accurately 
transmit  &  receive 

7 

The  SOI  TFT  Federation  must 
populate  the  Common  Operation 
Picture  (COP)  with  simulated 
entities. 

The  SOI  TFT  federation  must  provide  Situational  Awareness  (SA)  on  Force  XXI 

Battle  Command  Brigade  and  Below  (FBCB2)  displays  in  B-kit  current  force  vehicles 
and  in  simulated  and  live  Advanced  Field  Artillery  Tactical  Data  Systems  (AFATDS.) 
SA  will  consist  of  position  and  identification  (Blue  and  Red)  for  all  simulated  entities. 
Appropriate  messages  must  be  transmitted  and  received  between  simulated  and  live 
entities  in  the  operational  environment.  Live  must  be  displayed  on  the  simulation 
window  and  simulated  must  be  displayed  on  the  Live  window. 

ACM  7-1 

Net  wo  rk  d  isp  lay  □  f  SA 

ACM7-2 

Position  data 

8 

The  SOI  TFT  federates  must 
communicate  among  themselves  via 
appropriate  protocols. 

Federates  must  also  be  able  to  communicate  among  themselves  via  DoD  High  Level 
Architecture  (HLA)  and  Federation  Object  Model  (FOM)  enumerations  or  Distributed 
Interactive  Simulation  (DIS)  protocols,  as  appropriate. 

ACM  8-1 

Federation  compliance 

A  C  M  S -2 

Federation  RT1  -  testing  threshold 
for  repeatability 

ACM8-3 

Federation  RTI  interface  Specs 

ACM84 

Federation  Messages  -  accurately 
transmit  &  receive 

9 

All  federation  output  data  must  be 
adequate,  credible  and  in  a  usable 
format. 

SOI  TFT  federation  output  data  must  be  properly  formatted,  adequate  and 

credible  for  use  in  After  Action  Report  (AAR),  analysis  and  follow-on  test  events 
(e.g..  Force  Development  Test  and  Experimentation  (FDTE),  Limited  User  Test  (LUT) 
and  FCS  Core  test  events)  across  the  entire  spectrum  of  valid  input  data. 

ACM9-1 

Reasonable  Federation  Output 

AC  MS -2 

Graphical  output  i  *j 

ACM9-3 

Output  Data  Format 

Traceability  Analyses 


Developed  three  sets  of  Traceability  Analyses  Matrices 

1.  Traced  Acceptability  Criteria  Metrics  (ACMs)  to  the  Assessment  Objectives  (AOs). 

2.  Traced  ACMs  to  the  307  Test  Requirements,  defined  by  the  Test  Manager. 

3.  Traced  ACMs  to  the  Intended  Uses  (lUs). 

Traceability  Analyses  Benefits 

1.  Helped  to  refine  and  shape  the  Acceptability  Criteria  (AC)  &  ACMs. 

2.  Resulted  in  identifying  213  Test  Requirements  as  being  “true”  M&S  Requirements. 

3.  The  M&S  Requirements  traceability  matrix  provided  the  essential  underpinning 
element  for  the  Accreditation  Assessment  methodology,  including  how  to  best 
present  the  traceability  and  support  of  the  V&V  evidence  in  a  top-level 
Accreditation  Assessment  summary  format. 

4.  The  lUs  traceability  matrix  served  as  useful  template  to  present  top-level 
Accreditation  Assessment  summary  in  the  most  test  related  context,  from  the 
customer’s  and  user’s  perspective,  supported  by  the  M&S  Requirements 
traceability  matrix. 

5.  The  ACMs  and  307  Test  Requirements  Traceability  Analysis  was  deemed  the  most 
helpful  and  “priceless”  by  both  the  V&V  and  Accreditation  Teams. 


Traceability  Matrices  were  the  “backbone”  of  the  Accreditation  Assessment  methodt3ogy! 


Excerpt  -  Summary  SOI  TFT  ACMs-307  Test  Rqts  Traceability  Analysis 


Acce  pta  b  i  1  ity 

TFT  ACM  # 

TFT  ACM  Descriptions 

T-RAM  (M&S  Rqts) 

Total  T-RAM 

■Criteria 

Rqts  Addressed 

Title 

T  errain 

ACM1  1 

Scalability:  Shall  p rovi d e/d i s p  1  ay  scaleable  terrain  resolution 

TFT273 ,  TFT279 

2 

representations  are 

from  low  to  high  as  appropriate  for  the  test  article  or  federation 

at  required 
resolution 

component  participating  in  SOI  TFT. 

ACM1  -2 

3-D:  Shall  provide/  display  maps  and  three-dimensional  views  of 
terrain  for  SOI  TFT  vignettes  upon  demand.  Maps  are  typically 
2-0 . 

TFT245,  TFT273 

2 

AC  Ml  -3 

Terrain  Representation:  Terrain  representation  must  include: 
comparable  elevation  and  feature  data;  compatible  data  formats 
and  coordinate  systems;  environmental  effects  and  necessary- 
elevation  interpolation 

IF  138 

1 

AC  M2  -1 

Configuration  identification.  Settle  on  the  federation 

■C  o  nfi  g  u  rati  o  n 

components,  to  include  name,  date  and  version  that  will  be  used 

Managed 

in  te st  a  1 1  o w  s uffi ci e nt  ti m e  fo r 

Federation 

pre-test  prep  [e.g.,  integration  and  i nte ro p e ra b i  1  ity  te sti n g  of 

components).  [Though  not  a  test  metric,  this  is  critical  issue 

Requirement  maps  to  Army  MiS 

that  shall  remain  here  until  resolved.  Federation  configuration 

policy,  AR  5—1  1  ,  not  to  T-FLAM. 

identification  is  VS. V  artifact  and  shall  be  mentioned  in  the 

accreditation  plan.] 

ACM2-2 

Configuration  control.  Enable  all  authorized  users  to  have  the 

Requirement  maps  to  Army  M 3 S 

current  federation  components  and  associated  documents 
available  at  all  times  so  that  none  are  working  with  out-of-date 
material. 

policy,  AR  5—1  1  ,  not  to  T-RAM. 

AC  M2  -3 

Configuration  status  accounting.  Ensure  that  no  changes  are 

Requirement  maps  to  Army  M-3.S 

made  to  the  federation  baseline  without  proper  review  and 
approval,  preserving  the  federation's  intended  purpose, 
schedule,  and  cost. 

policy,  AR  5—1  1  ,  not  to  T-RAM. 

ACM24 

Config  u  ration  h  i  story .  F  rovi  d  e  tra  ce  a  b  i  1  ity  fo  r  co  nfi  g  u  rati  o  n 

Requirement  maps  to  Army  M-3.S 

modifications  during  the  SOI  TFT  federation  life  cycle,  after  3 CE 
hand-off,  as  appropriate. 

policy,  AR  5-1  1  ,  not  to  T-RAM. 

“The  simulated 

ACM3-1 

Identification  of  Simulated  Entities  /  Objects:  Simulated  and  live 

TFT2fi7,  TFT2S-B,  TFT7, 

5 

entities  must 

entities  <;e.g.,  soldiers,  vehicles,  etc.)  and  neutral  objects  shall 

TFT294.  TFT21 6 

perform  and 

appear  and  must  be  correctly  identified  as  expected  for  the  SO"1 

interact  as  required 

TFT  vignettes. 

to  support  the  SOI 

AC  M3  -2 

Current  Force  and  Threat  Entities  and  SOI  Systems:  These 

TFT  1 47,  TFT126,  TFT1S4, 

21 

TFT. 

simulated  entities  (soldiers,  vehicles,  etc.)  shall  appear  and 

T  FT  1  27  TFT  1  23,  TFT42  , 

perform  as  expected  for  their  intended  use  based  on  the  SOI 

TFT  1  5,  TFT  1  EE,  TFT36, 

TFT  vignettes.  Entities  shall  also  display  the  right  level  of  fidelity, 

TFT  1 7,  TFT39,  TFT2S, 

complexity  and  level  of  detail  that  are  acceptable  for  its  intended 
usage. 

TFT2&,  TFT150,  TFT226, 

TFT  1  35,  TFT1  S4,  TFT 1  35, 

TFT  1  33,  TFT257 ,  TFT41 

All  federation  input 

ACM4-1 

VS.V  of  input  data:  Input  data  used  in  the  SOI  TFT  federation  will 

TFT296,  TFT251 

2 

data  must  be 

be  verified  and  validated,  and  accredited/certified  for  this  specific 

identified  and 

intended  used,  as  appropriate  mindful  of  the  components  and 

provided  as 

test  articles  classification  that  will  be  used  to  conduct  the  test. 

required  for  SOI 

[ATEC  PAM  73-21} 

TFT  and  certified  for 

ACM4-2 

Data  Certification  Letter:  Input  data  verification,  validation  and 

Requirement  maps  to  Army  M&S 

the  intended  use  by 

certification  are  required  to  be  substantiated  by  a  letter  certifying 

policy,  AR  5—1  1  ,  not  to  T-RAM. 

authoritative 

its  use.  This  letter  signed  by  the  data  proponent  organization's 

sou  rce  ts  } 

commander,  agency  head  or  authorized  designee  certifies  that 
this  data  is  appropriate  for  this  specific  intended  use.  providing 
the  scope  of  its  use  such  as  known  constraints,  assumptions, 
caveats,  classification,  etc. 

The  SOI  TFT 

ACM5-1 

No  delay:  l\Jo  perceivable  message  or  display  delay  or  data 

TFT  1  92,  TFT  1  O 

2 

Federation  must 

packet  loss  during  real  time  operation  to  include  messages 

run  in  real-time  to 

received  and  displayed  on  the  COP.  [Supports  information 

support  live  test. 

timeliness  .] 

ACM  5 -2 

Ground  Truth  Display:  No  abnormality  in  perceived  ground  truth 

TFT273,  TFT216,  TFT  1  2, 

5 

display,  the  degree  to  which  the  federation  matches/keeps  up 
with  the  live  test. 

TFT239,  TFT26S 

v&v 

Description 
Form  for  M&S 
federation, 
federates,  and 
support  tools 


■  Described  in  detail 
what  the  M&S 
component  brought 
to  table  and  how  it 
would  be  used, 
expected  final 
version,  etc. 

■  Forms  were 
completed  by  the 
developers. 

■  A  standard  form  was 
developed  to 
document  V&V 
information  for  all 
the  M&S  Federation, 
federates, 
components,  and 
support  tools. 


1 

Model/Simulation/Tool  Name 

4 

Provider  Org 

5POC  Name 

2Version 

3 

Release  Date 

6Phone 

7Email 

8Description/Capabilities 


Assumptions 


10Limitations 


Input  Data  Used  for  SOI  TFT 


11  Terrain  DB 

12  Description 

13  Data  Source 

14  File  Name 

15  Description 

16  Data  Source 

17  Embedded  Data 

18  Other  Data 

Output  Data  Available  for  SOI  TFT 

19  File  Name 

20  Description 

21  Other  Output  Data 

SOI  TFT  Federation  Specific  Data 

22 

Intended  Use  w/in 
Federation 

23 

Description 

24  Federation  Data 
Published 

25  Data 

Subscriber 

26  Data  Description 

27 

Federation  Data 
Subscribed 

28 

Data 

Publisher 

29 

Data  Description 

30  Tactical/Live 
Component  l/F 

31 

Data  Network 

32  Send/ 
Receive 

33 

Data  Description 

Other  Data  l/F 

Data  Network 

36  Send/ 
Receive 

37  Data  Description  1  4 

Example  of  Completed  SOI  TFT  V&V  Artifacts  Log 


■  Described  M&S 
requirement  in 
detail,  who 
performed  ,  at 
what  event  was 
data  collected  , 
verification 
method  &  status, 
ACMs  supported, 
procedure  and 
results. 

■  Completed  by 
the  V&V  Team. 

■  Form  provided 
standardized 
format  as  part  of 
V&V  process 
and 

documentation 
supporting  the 
M&S 

Accreditation  . 


Legend:  ™  Met(M) 

Risk  None 


Task  ID 

V&V  Activity  Log 

Log  # 

x-TFT  2 

4. 4. 5. 6 

Name  Verify  Implementation  of  Requirements 

Description 

Technical  Requirement: 

The  SOI  M&S  Federation  shall  provide  the  capability  to  create,  modify,  copy, 
and  delete  a  force-on-force  scenario. 

Requirement  Understanding: 

Use  OOS  Management  &  Control  Tool  (MCT) 

Requirement  Limitation 

N/A 

Verification  Criteria 

1 )  Demonstrate  create  scenario 

2)  Demonstrate  modify  scenario 

3)  Demonstrate  copy  scenario/  rename  scenario 

4)  Demonstrate  delete  scenario 

5)  Demonstrate  merge  scenario 

Performed  By 

Becky  Hill  |  Date  11/1/07 

Supporting  Test  Event  IE8  Date  10/25/07 

Verification  Method  Demonstration  Verification  Status 

Met 

Procedure 

Perform  independent  assessment  of  OOS-OF  (v##)  MCT  to  verify  scenario 
generation  capabilities. 

Results 

1)  Scenario  created  through  selection  of  “New”  under  the  “File”  menu; 
entities  /  units  can  be  added;  force  structure  created;  initial  conditions 
assigned  to  entities.  (See  attached:  Create  Scenario) 

2)  Any  loaded  scenario  can  be  modified  through  the  entity  “Status”  ,  the 
“Task  Organization”  or  the  “Mission  Editor”  windows.  (See  attached: 
Modify  Scenario  —  only  “Status”  window  is  shown) 

3)  Scenario  can  be  copied/renamed  via  Windows  copy/paste  or  via  the 
OOS  MCT  “Save  As”  option 

4)  Scenario  can  be  deleted  via  Windows  delete  or  via  the  OOS  MCT 
“Delete”  option  within  the  “Manage  Scenarios”  feature  (See  attached: 
Manage  Scenario) 

5)  Scenarios  may  be  merged  by  opening  them  simultaneously  in  the 

OOS  MCT  and  then  doing  a  “Save  As”  to  create  a  new  scenario  that  is 
inclusive  of  the  set  of  opened  scenarios. 

□  Met  Minus  (M-) 

□  Partially  Met  (PM) 

■  PM  Minus  (PM-) 

M  Not  Met  (NM) 

15 

*— 1  Not  Tested 

Low/Minor 

Medium/Moderate 

Major 

Catastrophic/High 

Results 

and 

Lessons  Learned 


16 


SOI  TFT  M&S  V&V  Artifacts’  Summary 


Verification  Activities 

Requirements  Implementation  Verification 

•  210/213  Requirements  Met 

•  3  Requirements  Partially  Met 

•  No  Impact  to  M&S  Federation  Ability  to 
Support  SOI  TFT 
Data  V&V 

•  No  Data  Certification  of  Unclassified  Data 

•  Data  Sources  Identified 

•  No  Adverse  Impacts  Due  to  Unclassified  Data 

•  Consistent  use  of  Data  (Terrain,  Configuration 
Files) 

Networks  /  Data  Interfaces 

•  All  Simulation  to  Live  and  Simulation  to 
Simulation  Interfaces  Successfully 
Demonstrated 


Validation  Activities 

•  Mission  Threads  (Call  For  Fire  (CFF)  and  Situational  Awareness  (SA)) 

•  Essential  Message  Content  (HLA,  JVMF,  AFATDS) 

•  Scenarios  are  Reflective  of  Operational  Context 

Overall  V&V  Status 

•  All  Required  M&S  Capabilities  Provided 

•  M&S  V&V  Activities  Documented  in  SOI  TFT  M&S  V&V  Report  (Draft  #2,  8  Feb  08) 

•  Outstanding  Issues  Sufficiently  Resolved  Since  Integration  Event  9  Regression  (IE9R)  Test  (21-22  Jan  08) 

•  No  Major  Issues  Outstanding  17 


Discriminator 

Total 
#  Req 

Requirements  Verification  Status  Following  Key 
Events 

IE8 

IE9 

IE9R 

V9  Dry  Runs 

M 

|PM 

NT 

M 

PM  NT 

M 

PM  NT 

M 

PM  NT 

Classification 

2 

1 

1 

1 

1 

2 

°l 

2 

°l 

Battlespace  Rep 

22 

6 

8 

8 

21 

1 

21 

1 

22 

ol 

Scenario  Generation 

35 

27 

8 

31 

1  |  3 

31 

-l|  0 

35 

o|  0 

Simulate/Emulate  ABCS 

17 

1 

16 

17 

17 

17 

Simulate/Emulate  SOI 

Sys 

10 

3 

7 

9 

1 

10 

0 

10 

0 

Control  Cells 

1 

i 

i 

1 

1 

Live  Player  Integ 

13 

2 

i 

10 

13 

13 

13 

Setup  and  Execution 

15 

3 

2 

10 

12 

3  1 

14 

1 

15 

ol 

Exercise  View 

27 

4 

2 

21 

24 

3I 

24 

3I 

26 

1 

Data  Collection 

65 

12 

4 

49 

53 

5I  7 

53 

8U  4 

63 

2I  0 

VV&A  Certification 

4 

1 

3 

3 

1 

3 

1 

4 

o| 

AAR 

2 

2 

2 

2 

2 1 

TOTAL 

213 

55 

23 

135 

187 

16|  10 

191 

18  4 

210 

3  |  0 

As  14Feb08  from  Becky  Hill,  V&V  Lead 


Top-level 
Accreditation 
Assessment 
(AA) 

Summary 


Accreditation  Assessment  Summary  Results  Matrix  for  M&S  Federation  Supporting  FCS  SOI  TFT 


Acceptability  Criteria  (AC)  Title 

TFT 

AC  M# 

TFT  ACM  Descriptions 

ACM 

Status 

AC 

Status 

Accreditation  Impact 
Statement 

Terrain  representations  are  at  required  resolution. 

ACM1-1 

Terrain  scalability 

ACM1-2 

Terrain  3— ID  display 

ACM1-3 

Terrain  representation 

SOI  TFT 
Test 

Readiness 

Review 


TFT 


ACM 


AC 


Accreditation  Impact 


Acceptability  Criteria  (AC)  Title 

TFT 

ACM  # 

TFT  ACM  Descriptions 

ACM 

Status 

AC 

Status 

ACM  1-1 

Terrain  scalability 

M 

Terrain  representations  are  at  required  resolution. 

ACM  1-2 

Terrain  3-D  display 

M 

M 

ACM  1-3 

Terrain  representation 

M 

ACM2-1 

Configuration  identification 

M 

Configuration  Managed  Federation. 

ACM2-2 

Configuration  control 

M 

M 

ACM2-3 

Configuration  status  accounting 

M 

ACM2-4 

Configuration  history 

M 

The  simulated  entities  must  perform  and  interact  as 

ACM3-1 

ID  of  simulated  entities  &  objects 

M 

required  to  support  the  SOI  TFT. 

ACM  3-2 

Representation  of  CF,  threat  entities  & 
SOI  systems 

M 

M 

All  federation  input  data  must  be  identified  and 
provided  as  required  for  SOI  TFT  and  certified  for  the 

ACM4-1 

V&V  of  input  data 

M 

M 

intended  use  by  authoritative  source(s). 

ACM4-2 

Data  certification  letter 

M 

SOI  TFT  Federation  must  run  in  real-time  to  support 
live  test. 

ACM  5-1 

No  message  delay 

M 

ACM  5-2 

No  ground  truth  delay 

M 

M 

The  SOI  TFT  Federation  must  sup  port/  in  temperate 
with  current  force  battle  command  networks. 

ACM  6-1 

Commo  &  interoperate  w/  existing 

Army  Fire  controls 

M 

ACM  6-2 

Fire  mission  -  PAM  flyout 

M 

M 

ACM  6-3 

Tactical  message  -  accurately  transmit 
&  receive 

M 

The  SOI  TFT  Federation  must  populate  the  Common 

ACM7-1 

Network  display  of  SA 

M 

■V  J| 

Operation  Picture  (COP)  with  simulated  entities. 

ACM  7-2 

Ftosition  data 

M 

M 

ACM8-1 

Federation  compliance 

M 

The  SOI  TFT  federates  must  communicate  among 

ACM  8-2 

Federation  RTI  -  testing  threshold  for 
repeatability 

M 

IWI 

themselves  via  appropriate  protocols. 

ACM  8-3 

Federation  RTI  interface  Specs 

M 

IVI 

ACM8-4 

Federation  Messages  -  accurately 
transmit  &  receive 

M 

ACM9-1 

Reasonable  Federation  Output 

M 

All  federation  output  data  must  be  adequate,  credible 
and  in  a  usable  format. 

ACM  9-2 

Graphical  output 

M 

M 

ACM  9-3 

Output  Data  Format 

M 

Legend: 


Risk 


Met(M)  Met  Minus  (M-) 

None  Low/Minor 


□  Partially  Met  (PM) 
Medium/Moderate 


I  PM  Minus  (PM-) 
Major 
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I  Not  Met  (NM) 

Catastrophic/High  Asof15Feb08 


Top-level 

AAM&S 

Intended 

Uses 

Summary 


SOI  TFT 
Test 

Readiness 

Review 


Accred 

SOI  TFT  M. 

itation  Assessment  (AA)  of 
&S  Federation  Intended  Use  (IU) 

Intended  Use 

IU  Metric 

AA  IU 

Metric 

Status 

AA  IU 
Status 

TFT 

Accepability 
Critieria  (AC) 

TFT  AC  Metrics 

Accreditation 

Impact 

Statement 

Accreditation  Assessment  (AA)  of 
M&S  Federation  Intended  Uses  (IU)  Supporting  SOI  TFT  TRR 


Intended  Use 

IU  Metric 

AA  IU 

Metric 

Status 

AA  IU 

SOI  TFT 

(IU) 

Status 

Accepability  Critieria  (AC) 

SOI  Systems 

M 

AC3,  AC6 

CF  Systems  with  and 
w/o  B-Kits 

M 

AC3,  AC6,  AC7 

Constructive 

Representation 

Aggregate  Entities 

M 

M 

AC3,  AC6,  AC7 

Threat  Systems 

M 

AC3,  AC5,  AC7 

WSMR  Terrain 

M 

AC1 

Interface  to  Live  entities  via  tactical  messages 

M 

AC3,  AC5,  AC6,  AC7 

Operational  context  for  SOI  test  cases 

M 

AC3,  AC4,  AC5,  AC6,  AC7 

Test  support 

M 

AC2,  AC4,  AC8,  AC9 

Legend:  ™  Met(M) 

Risk  None 


■  PM  Minus  (PM-) 
Major 


M  Not  Met  (NM) 
Catastrophic/High 
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As  of  25Jan08 


□  Met  Minus  (M-) 
Low/Minor 


□  Partially  Met  (PM) 
Medium/Moderate 


SOI  TFT  Accreditation  Lessons  Learned 


Operational  /  Implementation 

•  Participated  in  Integration  Events  and  Dry  Runs 

■  Developed  Traceability  Matrices 

■  Developed  Relevant  Acceptability  Criteria  and  Metrics 

■  Expanded  Intended  Uses  to  include  Metrics 

■  Leveraged  resources  and  W&A  documentation  where  possible 

Process 

•  Started  early  -  planning  &  working  with  complete  W&A  Team 

■  Engaged  all  stakeholders  early 

■  Coordinated  the  staffing  and  approval  requirements 

■  Developed  Accreditation  standardized  formats 

■  Refined  V&V  standard  forms  to  better  support  Accreditation 

■  Established  an  Accreditation  process  that  has  been  used  in 
subsequent  test  and  demonstration  events 

Both  Operational  /  Implementation  and  Process 

•  Encouraged  Team  work 

■  Used  collaborative  environment  -  made  sharing  data  easier 


Templates  of  Standardized  Formats  &  Forms  Developed  by  VV&A  Team 

Want  to  share  with  others  the  formats  and  forms  that  the  SOI  TFT 
VV&A  Team  developed  and  found  the  most  useful  to  achieve  our  goal. 


Accreditation  Formats  Developed 


■  Acceptability  Criteria  Traceability  to  M&S 
Requirements 


Traceability 
Vralysts  Matrix  fern 


■  Accreditation:  Acceptability  Criteria 
Assessment  Matrices 


■  Blank 

■  Preliminary  Readiness  Review  w/  Risk  Impact 

Column  g*- 

■  Readiness  Review 

Aocredtabon 
Assessment  Matrix 


■Accreditation:  Intended  Use  Assessment 
Matrices 

■  Blank 

■  Preliminary  Readiness  Review  w/  Risk  Impact  Column 
and  Traceability  Columns 

■  Readiness  Review  w /  Traceability  Columns 

0 

AA  Intended  Use 
Matrix 


V&V  Artifact  Forms  Developed 

■  V&V  M&S  Component  /  Tool  /  Support  Tool 
Description  Form 

■  To  be  completed  by  the  individual  developer 


V8V  M&S 

ipenert  Description  F 


■  V&V  Activity  Log 

■  To  be  completed  by  V&V  Team 


V8V  Activity  Log 
form 
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Disparity 

tslan^ 
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Don’t  put  yourselves  on  Disparity  Island! 


Oh  boy,  dinner! 

Yeah,  somebody  didn’t  verify  some  units, 
another  didn’t  val. . . 

Who  cares? 

Yeah,  we  get  to  bite  their  dust 
‘cause  in  VV&A  they  didn’t  spend  the  bucks. 


i,suo!)seno 
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Back-ups 
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SOI  TFT  Test  Goals 


The  TFT  primary  goals  are  to  integrate  the  SOI  systems,  demonstrate  SOI 
capabilities,  collect  data  to  address  assessment  objectives,  and  provide  objective 
data  necessary  for  leadership  to  determine  if  SOI  systems  are  ready  to  proceed 
to  FDTE  and  LUT.  In  preparation  for  TFT,  the  SOI  systems  are  integrated  into  a 
SOI  TFT  slice  of  a  current  force  Heavy  Brigade  Combat  Team  (HBCT) 
organization  structure;  this  slice  is  identified  as  the  HBCT(-)  and  is  described  in 
section  4.3.  The  integration  will  be  phase  driven  by  the  incremental  delivery  of 
SOI  systems,  availability  of  trained  soldiers,  and  the  incremental,  controlled 
build-up  of  systems.  Each  phase  will  add  increasing  complexity  leading  to  the 
final  test  configuration.  Once  integrated,  the  SOI  capabilities  will  be  tested  and 
demonstrated  with  a  representative  slice  of  the  HBCT  utilizing  soldiers  trained  on 
the  specific  systems  operations  in  a  field  environment  at  WSMR.  The  SOI 
capabilities  tests  and  demonstrations  will  be  recorded  per  the  appropriate  data 
requirements  so  that  the  AOs  can  be  assessed.  The  collected  data  along  with 
the  AO  analysis  results  will  be  provided  to  leadership  at  the  conclusion  of  TFT  for 
assessment  to  proceed  to  FDTE  and  LUT. 


Source:  Doc#  D786-1 1925-1  Draft  -  dated  5Feb07 
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Acronym 

Definition 

3CE 

Cross  Command  Collaboration  Effort 

3DVIZ 

Three  Dimensional  (3D)  Visualization 

AAR 

After  Action  Review 

ABCS 

Army  Battle  Command  Systems 

AC 

Acceptability  Criterion 

ACE 

Advanced  Collaborative  Environment 

ACM 

Acceptability  Criterion  Metric 

AEC 

Army  Evaluation  Center 

AFATDS 

Advanced  Field  Artillery  Tactical  Data  System 

AO 

Assessment  Objective(s) 

ARDS 

Acquisition,  Reporting  and  Display  System 

ASAS  L 

All  Source  Analysis  System  Light 

AT  AC MS 

Army  Tactical  Mission  System 

ATEC 

Army  Test  and  Evaluation  Command 

BCS3 

Battle  Command  Sustainment  and  Support  System 

B-Kit 

An  installation  kit  for  Group  B  equipment 

BLUFOR 

Blue  Forces 

C2 

Command  and  Control 

C4I 

Command,  Control,  Communications,  Computers,  and  Intelligence 

C4ISR 

C4I  Surveillance,  and  Reconnaissance 

CCN 

Common  Control  Node 

CF 

Current  Force 

CFF 

Call  For  Fire 

COP 

Common  Operating  Picture 

CTO 

Combined  Test  Organization 

CTTK-DRA 

C4ISR  Test  Tool  Kit  —  Data  Reduction  and  Analysis 

DAUVS 

Digitized  Army  USMTF/VMF  Stimulator 

DCARS 

Digital  Collection,  Analysis,  and  Review  System 

DTC 

Developmental  Test  Command 

E-IBCT 

Early  /  Enhance  Integrated  Brigade  Combat  Team 

DTP 

Detailed  Test  Plan 

ExCIS 

Extensible  (C4I)  Instrumentation  Suite 

FBCB2 

Future  XXI  Battle  Command  Battalion/Brigade  and  Below 

Ft  Bliss  HITS 

Ft.  Bliss  Homestation  Instrumentation  Tool  Suite 

FCS 

Future  Combat  Systems 

FDT&E 

Force  Development  Test  and  Experimentation 

FFID  Sim  Fac 

Future  Force  Integration  Directorate  Simulation  Facility 

FOM 

Federation  Object  Model 

Acronym 
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FSE 

FCS  Simulation  Environment 

GPCS 

Ground  Platform  Communication  System 

HBCT 

Heavy  Brigade  Combat  Team 

HLA 

High  Level  Architecture 

HMMWV 

High-Mobility  Multipurpose  Wheeled  Vehicle 

IBCT 

Integrated  Brigade  Combat  Team 

IE 

Integration  Event 

IHITS 

Initial  Homestation  Instrumentation  Test  Systems 

IMS 

Integrated  Master  Schedule 

IMT 1 

Integrated  Mission  Test  One  (1) 

IPT 

Integrated  Product  Team 

IPS&T 

Integrated  Phases,  Simulation  and  Test 

IRS 

Intelligence,  Surveillance,  and  Reconnaissance 

IS&T 

Integration,  Simulation  &Test 

IU 

Intended  Use 

IV&V 

Independent  Verification  and  Validation 

JTRS 

Joint  Tactical  Radio  System 

JVMF 

Joint  Variable  Message  Format 

LDAP 

Lightweight  Directory  Access  Protocol 

LDIF 

LDAP  Data  Interchange  Format 

LSI 

Lead  Systems  Integrator 

LUT 

Limited  User  Test 

MCS 

Mounted  Combat  System  /  Mobility  Computer  System/  Maneuver  Control  System 

M&S 

Modeling  and  Simulation 

MATREX 

Modeling  Architecture  for  Technology  and  Research  Experimentation 

MILES 

Multiple  Integrated  Laser  Engagement  System 

MLRS 

Multiple  Launch  Rocket  System 

MS&O 

Modeling,  Simulation  and  Operations 

MSEL 

Master  Scenario  Events  List 

MSO 

Modeling  and  Simulation  Office 

MULE 

Multi-function  Utility/Logistics  and  Equipment  Vehicle 

MUSE 

Multiple  UAV  Simulation  Environment 

N/A 

Not  Applicable 

NLOS-LS 

Non  Line  of  Sight  -  Launch  System 

OBE 

Overcome  by  Events 

OneSAF 

One  Semi-Automated  Forces 

OOS 

One  Semi-Automated  Force  Objective  System 

ORD 

Operational  Requirements  Document 

Acronym 
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os 

Operating  System 

OSTRD 

Objective  System  Test  Requirements  Document 

OTC 

Operational  Test  Command 

PEO 

Program  Executive  Office 

PM 

Program  Manager 

PMO 

Program  Management  Office 

PTRR 

Pre  Test  Readiness  Review 

RDECOM 

Research,  Development,  and  Engineering  Command 

REDFOR 

Red  Force 

(RICS)2 

Reconfigurable  Intelligent  Instrumentation  to  Control,  Collect,  Simulate,  &  Stimulate 

RPG 

Rocket  Propelled  Grenades 

RPWS 

Role  Player  Work  Station 

RTI 

Run  Time  Infrastructure 

S2F 

System  of  System  Simulation  Framework 

SA 

Situational  Awareness 

SOI 

Spin  Out  1 

SoS 

System  of  Systems 

SUGV 

Small  Unmanned  Ground  Vehicle 

TBD 

To  Be  Determined 

TORS 

Test  Conduct  and  Reporting  System 

TEMP 

Test  and  Evaluation  Master  Plan 

TFT 

Technical  Field  Test 

TOEL 

Time-Ordered  Events  List 

TRADOC 

Training  and  Doctrine  Command 

T-RAM 

Technical  Requirements  Alignment  Matrix 

TRR 

Test  Readiness  Review 

T-UGS 

Tactical  Unattended  Ground  Sensor 

USMTF 

Unified  Standard  Military  Text  Format 

U-UGS 

Urban  Unattended  Ground  Sensor 

V&V 

Verification  and  Validation 

VMF 

Variable  Message  Format 

VV&A 

Verification  Validation  and  Accreditation 

WSMR 

White  Sands  Missile  Range 

XML 

Extensible  Markup  Language 

